{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c3b3feff-775c-412d-bc63-39764d9c6f89",
   "metadata": {},
   "source": [
    "# 2.8 通过微调提升模型的准确度与效率\n",
    "\n",
    "## 🚄 前言\n",
    "\n",
    "前面的课程中，介绍了如何搭建一个答疑机器人，并尝试通过优化提示词和RAG过程、扩展插件等方式提升其能力。不过你可能会发现，随着流程变得复杂，应用处理节点（依赖大模型）的增多，导致响应也在变慢。\n",
    "\n",
    "在实际生产落地过程中，很多企业会尝试将一些任务单一的节点（如下图中执行NL2SQL的大模型），替换成参数量更小的模型，并对其进行微调，让它在特定的领域的表现可以对标更大参数规模的模型，来换取更快的响应速度，与更低的成本开销。\n",
    "\n",
    "NL2SQL任务样例：\n",
    "\n",
    "`模型输入：数据库的各种信息（各表信息、列信息） +  自然语言问题：有多少部门的负责人年龄超过56岁？`\n",
    "\n",
    "`期望模型输出：\"SELECT count(*) FROM head WHERE age > 56\"`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e10445b5",
   "metadata": {},
   "source": [
    "## 🍁 课程目标\n",
    "\n",
    "学完本课程后，你将能够：\n",
    "\n",
    "*   了解什么是模型微调\n",
    "    \n",
    "*   了解构建微调数据集的技巧\n",
    "    \n",
    "*   简单了解模型评测指标\n",
    "    \n",
    "*   通过案例，一步步体验如何微调"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef478926",
   "metadata": {},
   "source": [
    "## 📖 课程目录\n",
    "- [1. 模型小型化](#1------)\n",
    "  * [1.1 模型小型化方案](#11--------)\n",
    "    + [1.1.1 模型量化](#111-----)\n",
    "    + [1.1.2 模型剪枝](#112-----)\n",
    "    + [1.1.3 使用更小的模型](#113--------)\n",
    "  * [🤔 1.2 从全参微调到高效微调](#---12-----------)\n",
    "- [🛠️ 2. LoRA微调实践](#----2-lora----)\n",
    "  * [2.1 环境准备](#21-----)\n",
    "  * [2.2 微调数据准备](#22-------)\n",
    "  * [2.3 微调任务目标](#23-------)\n",
    "    + [2.3.1 尝试本地模型](#231-------)\n",
    "    + [2.3.2 评测指标](#232-----)\n",
    "    + [2.3.3 评测本地模型](#233-------)\n",
    "  * [2.4 微调模型](#24-----)\n",
    "  * [2.5 评测微调后的模型](#25---------)\n",
    "    + [2.5.1 尝试微调后的模型](#251---------)\n",
    "    + [2.5.2 评测微调后的模型](#252---------)\n",
    "  * [2.6 参数矩阵融合](#26-------)\n",
    "- [✅ 本节小结](#------)\n",
    "- [扩展阅读](#----)\n",
    "  * [开展微调任务前必须要关注的](#-------------)\n",
    "  * [微调数据集构建策略](#---------)\n",
    "- [🔥 课后小测验](#--------)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42749ae2-78cc-4eec-a33f-77c5e5996e35",
   "metadata": {},
   "source": [
    "## 1. 模型小型化\n",
    "\n",
    "在2.5节构造了一个查询员工信息的工具，需要进行NL2SQL（将问题转化为查询数据库的SQL语句），这里来横向对比一下`qwen-plus`和`qwen2.5-1.5b-instruct`的SQL查询效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5cd002b5-9fa6-4d14-9484-94d1f49a698f",
   "metadata": {
    "ExecutionIndicator": {
     "show": false
    },
    "execution": {
     "iopub.execute_input": "2024-11-29T03:46:25.581584Z",
     "iopub.status.busy": "2024-11-29T03:46:25.581428Z",
     "iopub.status.idle": "2024-11-29T03:46:27.624783Z",
     "shell.execute_reply": "2024-11-29T03:46:27.624324Z",
     "shell.execute_reply.started": "2024-11-29T03:46:25.581566Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "╔══════════╦══════════════════════════════════════════════════════════════╦══════════════════════════════════════════╗\n",
      "║          ║                          qwen-plus                           ║               qwen2.5-1.5b               ║\n",
      "╠══════════╬══════════════════════════════════════════════════════════════╬══════════════════════════════════════════╣\n",
      "║ 查询结果 ║               查询成功：教育部门共有66名员工。               ║ 查询失败：抱歉，我暂时无法回答您的问题。 ║\n",
      "║          ║                                                              ║                                          ║\n",
      "║ 回答耗时 ║                           0.997秒                            ║                 0.741秒                  ║\n",
      "║          ║                                                              ║                                          ║\n",
      "║ SQL语句  ║ SELECT COUNT(*) FROM employees WHERE department = '教育部门' ║                  ```sql                  ║\n",
      "║          ║                                                              ║  SELECT COUNT(*) AS number_of_employees  ║\n",
      "║          ║                                                              ║              FROM employees              ║\n",
      "║          ║                                                              ║      WHERE department = '教育部门';      ║\n",
      "║          ║                                                              ║                   ```                    ║\n",
      "╚══════════╩══════════════════════════════════════════════════════════════╩══════════════════════════════════════════╝\n"
     ]
    }
   ],
   "source": [
    "import os, time\n",
    "import prettytable as pt\n",
    "from openai import OpenAI\n",
    "\n",
    "\n",
    "qwen_plus = \"qwen2.5-72b-instruct\"\n",
    "qwen_2_5_1_5b = \"qwen2.5-1.5b-instruct\"\n",
    "\n",
    "\n",
    "# 定义一个员工查询函数\n",
    "def query_employee_info(query, llm):\n",
    "    client = OpenAI(\n",
    "        api_key=os.getenv(\"DASHSCOPE_API_KEY\"),\n",
    "        base_url=\"https://dashscope.aliyuncs.com/compatible-mode/v1\",\n",
    "    )\n",
    "\n",
    "    completion = client.chat.completions.create(\n",
    "        model=llm,\n",
    "        messages=[\n",
    "            {'role': 'system', 'content': '''你是一个擅长写SQL的人，你有一个表叫employees，记录公司的员工信息，这个表有department（部门）、name（姓名）、HR三个字段。\n",
    "             department中有教育部门、后勤部门、技术部门。\n",
    "             你需要根据用户输入生成sql语句进行查询,你一定不能生成sql语句之外的内容，也不要把```sql```这个信息加上。'''},\n",
    "            {'role': 'user', 'content': query}])\n",
    "    SQL_output = completion.choices[0].message.content\n",
    "    if SQL_output == \"SELECT COUNT(*) FROM employees WHERE department = '教育部门'\":\n",
    "        return SQL_output, \"查询成功：教育部门共有66名员工。\"\n",
    "    if SQL_output == \"SELECT HR FROM employees WHERE name = '张三'\":\n",
    "        return SQL_output, \"查询成功：张三的HR是李四。\"\n",
    "    if SQL_output == \"SELECT department FROM employees WHERE name = '王五'\":\n",
    "        return SQL_output, \"查询成功：王五的部门是后勤部。\"\n",
    "    else:\n",
    "        return SQL_output, \"查询失败：抱歉，我暂时无法回答您的问题。\"\n",
    "\n",
    "# 初始化展示表格\n",
    "table = pt.PrettyTable([\"\", \"qwen-plus\", \"qwen2.5-1.5b\"])\n",
    "sql_row, ans_row, time_row, empty_row = [\"SQL语句\"], [\"查询结果\"], [\"回答耗时\"], ['','', '']\n",
    "\n",
    "# 设置表格“qwen_plus” 一列\n",
    "current_time = time.time()\n",
    "ans = query_employee_info(\"教育部门有几个人\", qwen_plus)\n",
    "sql_row.append(ans[0])\n",
    "ans_row.append(ans[1])\n",
    "time_row.append(str(round(time.time() - current_time, 3)) + '秒')\n",
    "\n",
    "# 设置表格“qwen2.5-1.5b” 一列\n",
    "current_time = time.time()\n",
    "ans = query_employee_info(\"教育部门有几个人\", qwen_2_5_1_5b)\n",
    "sql_row.append(ans[0])\n",
    "ans_row.append(ans[1])\n",
    "time_row.append(str(round(time.time() - current_time, 3)) + '秒')\n",
    "\n",
    "table.add_rows((ans_row, empty_row, time_row, empty_row, sql_row))\n",
    "table.set_style(pt.DOUBLE_BORDER)\n",
    "print(table)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8f897ea-208c-40fe-8459-b4614ebdd11a",
   "metadata": {},
   "source": [
    "可以看到，模型的参数减少到1.5B(billion)后，NL2SQL任务的答案虽然很接近了，但仍然有调教改进的空间。同时模型参数变小后也有多个显著优点：占用内存降低、使用成本降低、响应时间变快。总而言之就是能提高SLA，还能省钱。（如果正确率能一致）\n",
    "\n",
    "通俗地讲，一个通用的大模型很全能，就像旗舰手机一样，在摄像、处理器等几乎所有领域都很优秀，适合所有人但价格昂贵。而大部分用户的预算并不能负担旗舰手机，往往会选择能满足自己需求并且价格更加便宜的手机。\n",
    "\n",
    "这种花钱花在刀刃上的场景在实际应用中非常常见，所以能将模型小型化的解决方案是非常有必要的，那让我们来看一下有哪些可行方案。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "444130eb-6d0a-4305-9cb8-f46101a3694a",
   "metadata": {},
   "source": [
    "### 1.1 模型小型化方案\n",
    "\n",
    "由于`qwen-plus`的模型参数规模未公开，暂且假设它有70B的参数规模。\n",
    "\n",
    "**任务目标：将模型参数缩减至1B左右的规模，即原来的1/70（1.5%）。**\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bf4ca666-7705-45db-9677-3359f9e58c44",
   "metadata": {},
   "source": [
    "#### 1.1.1 模型量化\n",
    "\n",
    "模型量化是一种最简单，也是如今很常用的模型压缩技术，核心就是将网络中的所有参与计算的值都降低精度，近似表示结果。图中这里展示了如何将FP32（32bit）的数使用INT4（4 bit）进行近似。\n",
    "\n",
    "<img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/yBRq1MA2ppZKqdv1/img/de9d3691-2dd9-49c7-82df-36525eea1ea3.png\" width=\"300\">\n",
    "\n",
    "模型量化后相当于把模型规模减少至1/8(12.5%)，虽然这已经是很了不起的压缩了，但并不能满足需求。不过有趣的是虽然只使用了12.5%的资源，但对模型的精度影响非常小（ <2%）。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c116763e-bf03-47bf-9a00-a66c2adde7b3",
   "metadata": {},
   "source": [
    "#### 1.1.2 模型剪枝\n",
    "\n",
    "这里我们可以很形象地把模型比作一棵树，我们有两种修剪枝叶、减少模型参数的方法：\n",
    "\n",
    "参数剪枝，像是从树上摘掉一些不重要的树叶，虽然对每根树枝都影响不大，但减掉的树叶总体会很多。\n",
    "\n",
    "结构剪枝，像是修剪掉不重要的整根树枝，而不是单独的树叶。\n",
    "\n",
    "<img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/yBRq1MA2ppZKqdv1/img/bc6675a9-cc6b-4fb9-886c-6f7ca20c6019.png\" width=\"500\">\n",
    "\n",
    "\n",
    "不过非常遗憾的是模型剪枝在基本不影响模型性能的前提下，最多只能减少一半左右（50%）的参数。所以模型剪枝也无法满足我们的要求。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d560dffa-b559-4775-8760-4f55c8bdc910",
   "metadata": {},
   "source": [
    "#### 1.1.3 使用更小的模型\n",
    "\n",
    "综合来看，精简原有模型（qwen-plus）很难达到预期的目标（参数减少至1.5%）。\n",
    "\n",
    "也许我们应该换个思路，是否可以直接使用参数规模符合要求的小参数模型？比如使用`Qwen2.5-0.5B`或者`Qwen2.5-1.5B`来替代原来的`qwen-plus`。但使用小参数模型相对于大参数模型在NL2SQL任务上能力的不足该如何弥补呢？\n",
    "\n",
    "**我们可以通过模型微调来定向提升模型在NL2SQL任务的能力。**模型微调可以在保持模型相对轻量化的同时，显著提升在某些特定任务上的准确度和效率。\n",
    "\n",
    "微调的准备一般分为以下几步：\n",
    "\n",
    "1.  **选择被微调模型**：本节中将以`qwen2.5-1.5b-instruct`为例作为被微调的模型。\n",
    "    \n",
    "2.  **准备微调数据集**：本节已经准备好了数据集。（关于数据集构建更多的介绍请前往拓展阅读）\n",
    "    \n",
    "3.  **选择微调方法**：根据任务需求与可用硬件配置选择不同的模型微调方法，但大致分为全参微调和高效微调两种方式，接下来我们将进一步探究微调方法的选择。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a46020f-7d6b-4e79-92a8-f7c6a977ddb5",
   "metadata": {},
   "source": [
    "### 🤔 1.2 从全参微调到高效微调\n",
    "\n",
    "在开始微调前，先粗略计算一下微调`qwen2.5-1.5b-instruct`时需要的显存资源。\n",
    "\n",
    "$模型参数内存=1,500,000,000×4 字节=6,000,000,000 字节=5.73 GB$（一个浮点数4字节）\n",
    "\n",
    "一般对模型进行全参微调时，大概需要模型参数内存的7~8倍，也就是**46GB**的显存。这个显存占用基本上超过了大多数显卡的配置，也超过了我们后面要用到的GPU实验环境显存。\n",
    "\n",
    "想象一下如果模型规模是30B、60B，对显存的要求将非常高。那有没有什么办法可以节省显存使用呢？\n",
    "\n",
    "当然有，业界研究出了很多高效微调 PEFT（Parameter-Efficient Fine-Tuning）技术。PEFT的核心思想就是在少训练甚至不训练被微调模型参数的情况下，达到与全参微调相似的效果。\n",
    "\n",
    "在PEFT的发展过程中，代表方法有：\n",
    "\n",
    "1.   Freeze，该方法是最早的PEFT方法。它在微调时冻结模型的大部分参数，仅训练模型中的小部分参数（比如最后几层神经网络），来快速适应特定任务的需求。但实践证明Freeze的微调效果欠佳。\n",
    "    \n",
    "2.  Adapter Tuning，该方法会在原有的模型架构上，在某些位置之间插入Adapter层，微调时模型原有参数不会被训练，只训练这些Adapter层，而原先的参数不会参与训练。  \n",
    "    该方法可以在只额外增加3.6%参数的情况下达到与全参数微调相似的效果。但它的实现比较复杂，需要针对被微调模型的结构进行定制，无法标准化，“手工感”比较强。\n",
    "\n",
    "    <img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/yBRq1MA2ppZKqdv1/img/f22b010b-6d6c-447c-8cb6-83b821f011bd.png\" width=\"600\">\n",
    "\n",
    "    \n",
    "3.  LoRA（Low-Rank Adaptation）是如今最普遍使用的PEFT方法。该方法通过引入一组低秩矩阵来调整被微调模型的权重，从而不改变模型结构，也不新增神经网络层。  \n",
    "    微调时只额外引入的低秩矩阵`A * B`，使得微调的参数量下降`2 * r / d`倍。一般的`d = 1024、r = 16`，这样微调参数量是被训练模型的3.1%。而微调效果经常能超过全参微调。  \n",
    "    <table><tr><td bgcolor=white>\n",
    "    <img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/oJGq765B3YDrnAKe/img/b2e2caf3-e8da-4ac8-b57d-abbb05f932c4.png\" width=\"800\"></tr></table>\n",
    "\n",
    "    如果你对LoRA的更多细节感兴趣，可以阅读：[_LoRA: Low-Rank Adaptation of Large Language Models_](https://arxiv.org/abs/2106.09685)\n",
    "    \n",
    "\n",
    "现在最常用的微调方法就是LoRA及其相关衍生技术，因为它相对于其他高效微调方法有非常多的优点：\n",
    "\n",
    "1.  LoRA直接作用于模型最底层的矩阵运算，通用性强，不需要像Adapter Tuning一样必须了解被微调模型的结构。\n",
    "    \n",
    "2.  微调的成果只存储在\"微调参数矩阵\"中，不会破坏被微调模型的结构也不会改变被微调模型参数。\n",
    "    \n",
    "3.  参数集成与分离实现方便，便于开发者快速实现和集成。\n",
    "    \n",
    "\n",
    "本文也将以LoRA微调来开展微调实验。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c80e04c1-4ba0-44a2-b814-9c8391e7916f",
   "metadata": {},
   "source": [
    "## 🛠️ 2. LoRA微调实践\n",
    "\n",
    "实践目标：使用准备好的模型与NL2SQL微调数据集进行微调实践，观察微调前后，模型在NL2SQL能力的提升。\n",
    "\n",
    "NL2SQL任务样例：（数据库的信息对模型输出是必要的，它包括表名、列名、主键、外键等信息，这些信息帮助模型正确地组织SQL语句）\n",
    "\n",
    "`模型输入：数据库的各种信息（各表信息、列信息） +  自然语言问题：有多少部门的负责人年龄超过56岁？`\n",
    "\n",
    "`期望模型输出：\"SELECT count(*) FROM head WHERE age > 56\"`\n",
    "\n",
    "本课程使用[ms-swift](https://github.com/modelscope/ms-swift/tree/main)（Modelscope Scalable lightWeight Infrastructure for Fine-Tuning）框架，它是魔搭社区专门为模型训练开发的开源框架，该框架支持350+ LLM和90+ MLLM（多模态大模型）的训练(预训练、微调、对齐)、推理、评测和部署。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f83fb7e-22e0-41b7-8471-74dfc576a569",
   "metadata": {},
   "source": [
    "### 2.1 环境准备\n",
    "\n",
    "由于模型微调对硬件性能要求较高，因此你需要使用PAI的交互式建模（DSW）重新创建一个包含GPU的实例，让你可以更快地完成微调任务。\n",
    "\n",
    "请在另一个窗口打开课程章节“1\\_0\\_计算环境准备”的内容作为参考，环境准备操作可以参考[计算环境准备](https://edu.aliyun.com/course/3130200/lesson/343310285)中的**步骤一：创建PAI DSW实例**，不同之处在于：\n",
    "\n",
    "1.  你需要给新的实例命名，比如acp\\_gpu，你需要确保这个名字不与运行2.7之前课程的实例重名；\n",
    "    \n",
    "2.  **资源规格**选择免费试用页签中的**ecs.gn7i-c8g1.2xlarge**，不用担心，你领取到的免费额度足够支持你完成2.7与2.8的课程；\n",
    "    \n",
    "3.  **镜像**选择**modelscope:1.18.0-pytorch2.3.0-gpu-py310-cu121-ubuntu22.04**（需要将“镜像配置”->”芯片类型“切换为GPU）。\n",
    "    \n",
    "4.  **当实例状态为运行中时，重复**[**计算环境准备**](https://edu.aliyun.com/course/3130200/lesson/343310285)**中的步骤二，获取大模型ACP课程的代码。**\n",
    "    \n",
    "5.  当完成以上步骤后，请在你新创建的GPU实例中再重新打开本章节，继续学习以下内容。\n",
    "    \n",
    "\n",
    "> 如果你在本地没有GPU环境，不建议你在本地学习本课程，代码的运行可能会失败。\n",
    "\n",
    "6.  安装以下依赖：\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fbab0ae-cda5-4aea-9b03-6c8fa931a21b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 你需要安装以下依赖\n",
    "!pip install accelerate==1.0.1 rouge-score==0.1.2 nltk==3.9.1 ms-swift[llm]==2.4.2.post2 evalscope==0.5.5rc1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c2b7f84-d7b4-4663-8da0-2c001900470f",
   "metadata": {},
   "source": [
    "7.  把模型下载到本地\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f78e0053-3919-43b7-8906-e9d6cef4b284",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "execution": {
     "iopub.execute_input": "2024-11-29T03:46:32.476249Z",
     "iopub.status.busy": "2024-11-29T03:46:32.475981Z",
     "iopub.status.idle": "2024-11-29T03:46:59.936944Z",
     "shell.execute_reply": "2024-11-29T03:46:59.936417Z",
     "shell.execute_reply.started": "2024-11-29T03:46:32.476230Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading [config.json]: 100%|███████████████| 660/660 [00:00<00:00, 1.54kB/s]\n",
      "Downloading [configuration.json]: 100%|███████| 2.00/2.00 [00:00<00:00, 4.05B/s]\n",
      "Downloading [generation_config.json]: 100%|██████| 242/242 [00:00<00:00, 610B/s]\n",
      "Downloading [LICENSE]: 100%|███████████████| 11.1k/11.1k [00:00<00:00, 22.9kB/s]\n",
      "Downloading [merges.txt]: 100%|████████████| 1.59M/1.59M [00:00<00:00, 1.94MB/s]\n",
      "Downloading [model.safetensors]: 100%|██████| 2.88G/2.88G [00:10<00:00, 289MB/s]\n",
      "Downloading [README.md]: 100%|█████████████| 4.79k/4.79k [00:00<00:00, 11.4kB/s]\n",
      "Downloading [tokenizer.json]: 100%|████████| 6.71M/6.71M [00:00<00:00, 7.90MB/s]\n",
      "Downloading [tokenizer_config.json]: 100%|█| 7.13k/7.13k [00:00<00:00, 10.8kB/s]\n",
      "Downloading [vocab.json]: 100%|████████████| 2.65M/2.65M [00:00<00:00, 3.39MB/s]\n"
     ]
    }
   ],
   "source": [
    "!mkdir ./model \n",
    "!modelscope download --model qwen/Qwen2.5-1.5B-Instruct --local_dir './model'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fec0ca11-cd40-458c-81eb-35d2d206a323",
   "metadata": {},
   "source": [
    "打开model文件夹，就可以看到Qwen2.5-1.5B-Instruct的模型文件了。\n",
    "\n",
    "<img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/ZWGl0wWZx7kDO34Y/img/53b3cb42-b307-4ec2-85ec-54ab5580e096.webp\" width=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b00b16b5-f6a2-4186-99aa-db321d4cf017",
   "metadata": {},
   "source": [
    "### 2.2 微调数据准备\n",
    "\n",
    "这里已经提前准备好了一份线上教育公司相关数据库问题查询的数据集，其中大概500+作为训练集，100+作为测试集。在`resources/2_4/data.zip`文件，你需要运行以下命令进行解压：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28f07e26-5e15-40fe-aa1e-f0c906b49ca5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!unzip -u resources/2_4/data.zip -d resources/2_4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5842076d-c12e-4aac-aab1-909e19db0d87",
   "metadata": {},
   "source": [
    "解压后的data文件夹在resources/2\\_4中，打开data文件夹可以看到\n",
    "\n",
    "<img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/ZWGl0wWZx7kDO34Y/img/a0c760ab-6ba2-475a-9b0e-f2ac16514824.png\" width=\"300\">\n",
    "\n",
    "其中train.jsonl为训练集文件，test.jsonl为测试集文件。\n",
    "\n",
    "这里介绍在模型训练时的数据分类：\n",
    "\n",
    "*   **训练集**：用于在**训练阶段**进行模型参数的训练。\n",
    "    \n",
    "*   **验证集**：用于在**训练阶段**评估模型训练的效果，从而调整模型的超参数（如学习率、退火策略等）和监控过拟合情况。\n",
    "    \n",
    "*   **测试集**：用于在**训练阶段完成后**评估模型训练的效果。本课程提供的实验数据中，测试集涉及的数据库均没有在训练、验证数据集中出现过。\n",
    "    \n",
    "\n",
    "> 训练集文件会被按照4:1的比例拆分为训练集与验证集，测试集文件则对应测试集。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9ffdcd2-0e13-4639-a723-50d759b12009",
   "metadata": {},
   "source": [
    "### 2.3 微调任务目标\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad98a9a5-5f33-432c-9b95-a08518c0a56e",
   "metadata": {},
   "source": [
    "#### 2.3.1 尝试本地模型\n",
    "\n",
    "在标准模型上运行一次NL2SQL任务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "0d2fe667-9402-4b4c-bf44-c46c372d1881",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-11-29T03:47:00.132834Z",
     "iopub.status.busy": "2024-11-29T03:47:00.132611Z",
     "iopub.status.idle": "2024-11-29T03:47:26.514873Z",
     "shell.execute_reply": "2024-11-29T03:47:26.514383Z",
     "shell.execute_reply.started": "2024-11-29T03:47:00.132816Z"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO:swift] Successfully registered `/usr/local/lib/python3.10/site-packages/swift/llm/data/dataset_info.json`\n",
      "[INFO:swift] Loading the model using model_dir: ./model\n",
      "[INFO:swift] model_kwargs: {'device_map': 'cpu'}\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "template_type: qwen2_5\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO:swift] model.max_model_len: 32768\n",
      "Starting from v4.46, the `logits` model output will have the same type as the model (except at train time, where it will always be FP32)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "输入：---------------------------------------------------------------------------------------------\n",
      "  #背景#\n",
      "    数据库信息:{'column_names': [[-1, '*', 'text'], [0, 'region id', 'number'], [0, 'region name', 'text'], [1, 'country id', 'text'], [1, 'country name', 'text'], [1, 'region id', 'number'], [2, 'department id', 'number'], [2, 'department name', 'text'], [2, 'manager id', 'number'], [2, 'location id', 'number'], [3, 'job id', 'text'], [3, 'job title', 'text'], [3, 'min salary', 'number'], [3, 'max salary', 'number'], [4, 'employee id', 'number'], [4, 'first name', 'text'], [4, 'last name', 'text'], [4, 'email', 'text'], [4, 'phone number', 'text'], [4, 'hire date', 'time'], [4, 'job id', 'text'], [4, 'salary', 'number'], [4, 'commission pct', 'number'], [4, 'manager id', 'number'], [4, 'department id', 'number'], [5, 'employee id', 'number'], [5, 'start date', 'time'], [5, 'end date', 'time'], [5, 'job id', 'text'], [5, 'department id', 'number'], [6, 'location id', 'number'], [6, 'street address', 'text'], [6, 'postal code', 'text'], [6, 'city', 'text'], [6, 'state province', 'text'], [6, 'country id', 'text']], 'foreign_keys': [[5, 1], [20, 10], [24, 6], [28, 10], [29, 6], [25, 14], [35, 3]], 'primary_keys': [1, 3, 6, 10, 14, 25, 30], 'table_names': ['regions', 'countries', 'departments', 'jobs', 'employees', 'job history', 'locations']}    \n",
      "  #受众#\n",
      "    Mysql数据库    \n",
      "  #输出#\n",
      "    只输出SQL查询语句    \n",
      "  #目的#\n",
      "    将问题\"What is all the information about employees who have never had a job in the past?\"转化为SQL查询语句'\n",
      "期望输出：-----------------------------------------------------------------------------------------\n",
      "  SELECT * FROM employees WHERE employee_id NOT IN (SELECT employee_id FROM job_history)\n",
      "输出：---------------------------------------------------------------------------------------------\n",
      "```sql\n",
      "SELECT *\n",
      "FROM employees\n",
      "WHERE employee_id NOT IN (\n",
      "    SELECT e.employee_id\n",
      "    FROM employees e\n",
      "    JOIN job_history jh ON e.employee_id = jh.employee_id\n",
      ")\n",
      "```\n",
      "END：---------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "from swift.llm import (\n",
    "    get_model_tokenizer, get_template, inference, ModelType,\n",
    "    get_default_template_type, inference_stream\n",
    ")\n",
    "import torch\n",
    "\n",
    "#你可以根据你的需要修改query（模型输入）\n",
    "query = \"  #背景#\\n\\\n",
    "    数据库信息:{'column_names': [[-1, '*', 'text'], [0, 'region id', 'number'], [0, 'region name', 'text'], [1, 'country id', 'text'], [1, 'country name', 'text'], [1, 'region id', 'number'], [2, 'department id', 'number'], [2, 'department name', 'text'], [2, 'manager id', 'number'], [2, 'location id', 'number'], [3, 'job id', 'text'], [3, 'job title', 'text'], [3, 'min salary', 'number'], [3, 'max salary', 'number'], [4, 'employee id', 'number'], [4, 'first name', 'text'], [4, 'last name', 'text'], [4, 'email', 'text'], [4, 'phone number', 'text'], [4, 'hire date', 'time'], [4, 'job id', 'text'], [4, 'salary', 'number'], [4, 'commission pct', 'number'], [4, 'manager id', 'number'], [4, 'department id', 'number'], [5, 'employee id', 'number'], [5, 'start date', 'time'], [5, 'end date', 'time'], [5, 'job id', 'text'], [5, 'department id', 'number'], [6, 'location id', 'number'], [6, 'street address', 'text'], [6, 'postal code', 'text'], [6, 'city', 'text'], [6, 'state province', 'text'], [6, 'country id', 'text']], 'foreign_keys': [[5, 1], [20, 10], [24, 6], [28, 10], [29, 6], [25, 14], [35, 3]], 'primary_keys': [1, 3, 6, 10, 14, 25, 30], 'table_names': ['regions', 'countries', 'departments', 'jobs', 'employees', 'job history', 'locations']}\\\n",
    "    \\n  #受众#\\n\\\n",
    "    Mysql数据库\\\n",
    "    \\n  #输出#\\n\\\n",
    "    只输出SQL查询语句\\\n",
    "    \\n  #目的#\\n\\\n",
    "    将问题\\\"What is all the information about employees who have never had a job in the past?\\\"转化为SQL查询语句'\"\n",
    "\n",
    "# 获得模型模板\n",
    "model_type = ModelType.qwen2_5_1_5b_instruct\n",
    "model_id_or_path = \"./model\"\n",
    "template_type = get_default_template_type(model_type)\n",
    "print(f'template_type: {template_type}')  # template_type: qwen2_5\n",
    "# 初始化模型\n",
    "kwargs = {}\n",
    "model, tokenizer = get_model_tokenizer(model_type, torch.float32, model_id_or_path=model_id_or_path, model_kwargs={'device_map': 'cpu'}, **kwargs)\n",
    "model.generation_config.max_new_tokens = 128\n",
    "template = get_template(template_type, tokenizer)\n",
    "# 执行推理\n",
    "response, history = inference(model, template, query)\n",
    "# 格式化输出\n",
    "print('输入：---------------------------------------------------------------------------------------------')\n",
    "print(query)\n",
    "print('期望输出：-----------------------------------------------------------------------------------------')\n",
    "print('''  SELECT * FROM employees WHERE employee_id NOT IN (SELECT employee_id FROM job_history)''')\n",
    "print('输出：---------------------------------------------------------------------------------------------')\n",
    "print(response)\n",
    "print('END：---------------------------------------------------------------------------------------------')\n",
    "# 缓存清理\n",
    "del model,template,tokenizer;import gc;_=gc.collect()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40bb7dbc-99a1-45d0-a141-7c5ad9b81c6d",
   "metadata": {},
   "source": [
    "#### 2.3.2 评测指标\n",
    "\n",
    "不同类型的任务评测指标有显著差异，比如：\n",
    "\n",
    "1.  **分类任务**：\n",
    "    \n",
    "    *   准确率（Accuracy）：正确预测的比例。\n",
    "        \n",
    "    *   精确度（Precision）、召回率（Recall）与F1分数（F1 Score）：用于衡量二分类或多分类问题中正类别的识别效果。\n",
    "        \n",
    "2.  **文本生成任务**：\n",
    "    \n",
    "\n",
    "    *   BLEU (Bilingual Evaluation Understudy) 分数：主要用于机器翻译等自然语言处理任务中，通过比较候选翻译与一个或多个参考翻译之间的n-gram重叠来计算得分。\n",
    "        \n",
    "    *   ROUGE (Recall-Oriented Understudy for Gisting Evaluation) 分数：常用于自动摘要评价，它基于n-gram召回率、精确率以及F-measure。\n",
    "        \n",
    "    *   Perplexity (困惑度)：用来衡量概率分布模型预测样本的不确定程度；越低越好。\n",
    "    \n",
    "\n",
    "3.  **图像识别/目标检测**：\n",
    "    \n",
    "\n",
    "    *   Intersection over Union (IoU)：两个边界框相交部分面积与并集面积之比。\n",
    "        \n",
    "    *   mAP (mean Average Precision)：平均精度均值，广泛应用于物体检测任务中。\n",
    "    \n",
    "\n",
    "NL2SQL任务的目的是输出一条可正确执行的SQL语句， 所以可以用“SQL语句是否输出正确”和“SQL语句执行结果是否正确”两个标准来衡量。这里我们为了实验方便，只使用“**SQL语句是否输出正确**”来评测任务。\n",
    "\n",
    "这里再简单介绍一下ROUGE Scores (Recall-Oriented Understudy for Gisting Evaluation)：\n",
    "\n",
    "`rouge-n-(r/p/f)`: 这是基于词级别的n-gram匹配度量，其中`n=1`，意味着考虑单个词汇的重合情况；`n=2`，意味着考虑两个词汇组成的短语的重合情况；`n=L(long)`，意味着考虑一个模型输出与测试用例中最长公共子序列的重合情况。\n",
    "\n",
    "*   `rouge-n-r`：召回率（Recall），表示模型输出正确预测出测试用例的比例。\n",
    "    \n",
    "*   `rouge-n-p`：精确率（Precision），表示模型输出有多大部分也出现在测试用例中。\n",
    "    \n",
    "*   `rouge-n-f`：F1分数，是召回率和精确率的调和平均值，综合考量了两者的表现。\n",
    "    \n",
    "\n",
    "> 相关文章：ROUGE: A Package for Automatic Evaluation of Summaries (Chin-Yew Lin, 2004)\n",
    "\n",
    "`Rouge-L`分数可以评估输出的SQL语句有多少部分是有用的，因此它能大致反映出模型的能力。后续我们将以`Rouge-L`作为评估指标。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa4e8a2f-36fc-4db9-b3b6-fbf5d9f0716e",
   "metadata": {},
   "source": [
    "#### 2.3.3 评测本地模型\n",
    "\n",
    "这里使用ms-swift对任务进行评测。使用的评测模式为：general\\_qa。该评测模式默认会计算`Rouge-1/2/L`分数。更多信息请参考：[问答题格式数据集构建方法](https://evalscope.readthedocs.io/zh-cn/latest/advanced_guides/custom_dataset.html#qa)。\n",
    "\n",
    "按照ms-swift框架要求，测试集文件每条数据遵循以下格式：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d72ae5db-8e35-43f3-bf43-8986dc4a7e85",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "{\n",
    "    \"history\": [\"表示对话历史\"],\n",
    "    \"query\": \"融合了背景信息与输入问题\",\n",
    "    \"response\":  \"query对应的真实答案\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc3dbc0a-2564-4493-8900-fbe0a5112ad5",
   "metadata": {},
   "source": [
    "> 本微调教程没有涉及多轮对话的数据，因此history字段为`[]`。\n",
    "\n",
    "评测命令：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11a2b1f7-fb53-42c2-a619-70efa8d11135",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!mkdir -p eval_outputs\n",
    "%env LOG_LEVEL=WARNING\n",
    "!swift eval \\\n",
    "--model_id_or_path './model' \\\n",
    "--model_type 'qwen2_5-1_5b-instruct' \\\n",
    "--eval_dataset no \\\n",
    "--custom_eval_config 'resources/2_4/data/config_eval.json' \\\n",
    "--max_length -1 \\\n",
    "--system '' \\\n",
    "--infer_backend 'pt' \\\n",
    "--name 'pre_train_evaluation' \\\n",
    "--eval_output_dir './eval_outputs'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfcb1404-043a-4161-8646-68170868f5c5",
   "metadata": {},
   "source": [
    "\n",
    "完整参数列表请参考：[eval 命令行参数](https://swift.readthedocs.io/zh-cn/latest/Instruction/%E5%91%BD%E4%BB%A4%E8%A1%8C%E5%8F%82%E6%95%B0.html#id16)\n",
    "\n",
    "我们关心的两个模型`Rouge-L`分数如下：（每次评测结果会有一些细微差异，这是由于大模型生成的随意性导致的）\n",
    "\n",
    "|   |  qwen-plus  |  qwen2\\_5-1\\_5b-instruct  |\n",
    "| --- | --- | --- |\n",
    "|  rouge-l-r  |  0.5995804511766886  |   0.4730549779987757  |\n",
    "|  rouge-l-p  |  0.5671844066005733  |  0.3572425303637290  |\n",
    "|  rouge-l-f  |  0.5755829728881274  |  0.3907812431837991  |\n",
    "\n",
    "我们的目标便是让微调后的`qwen2.5-1.5b-instruct`模型在NL2SQL任务上的`Rouge-L`分数相当或超过`qwen-plus`。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e58a7315-5ab8-4055-a018-cf4992bb7ae8",
   "metadata": {},
   "source": [
    "### 2.4 微调模型\n",
    "\n",
    "在完成了模型的评测后，就可以开始对模型进行微调了。swift提供了操作简单的微调工具，你只需要传入训练集、模型路径等参数。同时swift框架在微调时默认使用LoRA方法，在命令中不需要额外声明。\n",
    "\n",
    "微调命令："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f4d71db9-540a-4d27-9813-cb7af9476d1a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%env CUDA_VISIBLE_DEVICES=0\n",
    "%env LOG_LEVEL=INFO\n",
    "!mkdir -p logs\n",
    "!swift sft \\\n",
    "--dataset 'resources/2_4/data/train.jsonl' \\\n",
    "--learning_rate '1e-4' \\\n",
    "--eval_steps '10' \\\n",
    "--batch_size '4' \\\n",
    "--model_type 'qwen2_5-1_5b-instruct' \\\n",
    "--max_length 2048 \\\n",
    "--model_id_or_path './model' \\\n",
    "--num_train_epochs 3"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2043020-c47e-45e0-9324-47c91aec4b51",
   "metadata": {},
   "source": [
    "微调的完整参数列表请参考：[sft 命令行参数](https://swift.readthedocs.io/zh-cn/latest/Instruction/%E5%91%BD%E4%BB%A4%E8%A1%8C%E5%8F%82%E6%95%B0.html#id8) \n",
    "\n",
    "在微调时，并非所有训练样本都会直接参与到模型参数的训练中，swift框架会以4:1的比例将训练样本划分为训练集与验证集。其中训练集会直接参与到模型参数的训练；验证集虽然不直接参与训练过程，但在模型训练时提供了实时评测的效果，通过设置`--eval_steps`参数指定每训练多少steps，进行评测与保存当前模型的checkpoint（文件夹内包含“微调参数矩阵”、日志等），这样swift框架可以保存在验证集上获得最佳评测效果的checkpoint，作为最佳模型（即日志中打印的best\\_model\\_checkpoint）。并且在训练结束时，并不会把“微调参数矩阵”与“预训练权重矩阵”合并。\n",
    "\n",
    "<table><tr><td bgcolor=white>\n",
    "<img src=\"https://alidocs.oss-cn-zhangjiakou.aliyuncs.com/res/oJGq765B3YDrnAKe/img/b2e2caf3-e8da-4ac8-b57d-abbb05f932c4.png\" width=\"800\"></tr></table>\n",
    "\n",
    "完整的loss和eval\\_loss信息保存在文件`output/qwen2_5-1_5b-instruct/v2-202xxxxx-xxxx/logging.jsonl`中。\n",
    "\n",
    "*   **如果你观察到loss在减少，eval\\_loss也在减少**，说明模型需要继续训练。你可以增加`--num_train_epochs`的值来增加模型的训练深度。  \n",
    "    _可以将当前的模型类比为一个处在上升期的运动员，通过不断的练习即可稳步提升比赛成绩。_\n",
    "    \n",
    "*   **如果你观察到loss在减少，eval\\_loss却在增加，**说明模型已经处于**过拟合**状态（过度拟合了训练数据中的潜在模式）。在LoRA微调中，可以通过增加`--lora_dropout`的值，抑制模型的**过拟合**倾向。  \n",
    "    _可以将当前的模型类比为一个只会背题库的学生，无法掌握题目背后的机理。_\n",
    "    \n",
    "*   **如果你观察到loss在增加，eval\\_loss也增加，**（不常见）说明模型训练失败，需要停止训练并排查失败原因。常见的原因有：\n",
    "    \n",
    "    *   学习率设置过大，导致模型参数剧烈震荡。\n",
    "        \n",
    "    *   训练数据有问题，例如模型输入和预期输出不存在任何逻辑关系。\n",
    "        \n",
    "    *   抑制模型过拟合的参数设置过大，导致模型无法学习新知识。\n",
    "        \n",
    "\n",
    "你可以在output/qwen2\\_5-1\\_5b-instruct/v2-202xxxxx-xxxxxx/images文件夹查看训练过程中的loss、eval\\_loss等指标的变化情况。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5db30f94-2a8b-417a-82d4-83a389a305f8",
   "metadata": {},
   "source": [
    "### 2.5 评测微调后的模型\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42e1cbdc-e972-4e5f-9528-5ee211cba8af",
   "metadata": {},
   "source": [
    "#### 2.5.1 尝试微调后的模型\n",
    "\n",
    "你可以运行以下代码，测试微调效果。  \n",
    "运行以下代码前请将`ckpt_dir`修改为在验证集表现最佳的微调模型路径。微调后一般会保存两个checkpoint文件，分别是best\\_model\\_checkpoint（在验证集表现最佳的微调模型）与last\\_model\\_checkpoint（最后一次保存的checkpoint）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "21e15777-e7c3-48a4-9ffb-5eca9049a4b0",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "execution": {
     "iopub.execute_input": "2024-11-29T03:58:42.452117Z",
     "iopub.status.busy": "2024-11-29T03:58:42.451790Z",
     "iopub.status.idle": "2024-11-29T03:58:56.845066Z",
     "shell.execute_reply": "2024-11-29T03:58:56.844606Z",
     "shell.execute_reply.started": "2024-11-29T03:58:42.452096Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO:swift] Loading the model using model_dir: ./model\n",
      "[INFO:swift] model_kwargs: {'device_map': 'cpu'}\n",
      "[INFO:swift] model.max_model_len: 32768\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "输入：---------------------------------------------------------------------------------------------\n",
      "  #背景#\n",
      "    数据库信息:{'column_names': [[-1, '*', 'text'], [0, 'region id', 'number'], [0, 'region name', 'text'], [1, 'country id', 'text'], [1, 'country name', 'text'], [1, 'region id', 'number'], [2, 'department id', 'number'], [2, 'department name', 'text'], [2, 'manager id', 'number'], [2, 'location id', 'number'], [3, 'job id', 'text'], [3, 'job title', 'text'], [3, 'min salary', 'number'], [3, 'max salary', 'number'], [4, 'employee id', 'number'], [4, 'first name', 'text'], [4, 'last name', 'text'], [4, 'email', 'text'], [4, 'phone number', 'text'], [4, 'hire date', 'time'], [4, 'job id', 'text'], [4, 'salary', 'number'], [4, 'commission pct', 'number'], [4, 'manager id', 'number'], [4, 'department id', 'number'], [5, 'employee id', 'number'], [5, 'start date', 'time'], [5, 'end date', 'time'], [5, 'job id', 'text'], [5, 'department id', 'number'], [6, 'location id', 'number'], [6, 'street address', 'text'], [6, 'postal code', 'text'], [6, 'city', 'text'], [6, 'state province', 'text'], [6, 'country id', 'text']], 'foreign_keys': [[5, 1], [20, 10], [24, 6], [28, 10], [29, 6], [25, 14], [35, 3]], 'primary_keys': [1, 3, 6, 10, 14, 25, 30], 'table_names': ['regions', 'countries', 'departments', 'jobs', 'employees', 'job history', 'locations']}    \n",
      "  #受众#\n",
      "    Mysql数据库    \n",
      "  #输出#\n",
      "    只输出SQL查询语句    \n",
      "  #目的#\n",
      "    将问题\"What is all the information about employees who have never had a job in the past?\"转化为SQL查询语句'\n",
      "期望输出：-----------------------------------------------------------------------------------------\n",
      "  SELECT * FROM employees WHERE employee_id NOT IN (SELECT employee_id FROM job_history)\n",
      "输出：---------------------------------------------------------------------------------------------\n",
      "SELECT * FROM Employees WHERE Employee_id NOT IN (SELECT DISTINCT Employee_id FROM Job_history)\n",
      "END：---------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "from swift.llm import (\n",
    "    get_model_tokenizer, get_template, inference, ModelType, get_default_template_type,\n",
    ")\n",
    "from swift.tuners import Swift\n",
    "import torch\n",
    "\n",
    "# 请在运行前修改ckpt_dir到正确的位置\n",
    "ckpt_dir = 'output/qwen2_5-1_5b-instruct/vx-xxx/checkpoint-xxx<请修改为lora微调后checkpoint位置>'\n",
    "# 你可以根据你的需要修改query（模型输入）\n",
    "query = \"  #背景#\\n\\\n",
    "    数据库信息:{'column_names': [[-1, '*', 'text'], [0, 'region id', 'number'], [0, 'region name', 'text'], [1, 'country id', 'text'], [1, 'country name', 'text'], [1, 'region id', 'number'], [2, 'department id', 'number'], [2, 'department name', 'text'], [2, 'manager id', 'number'], [2, 'location id', 'number'], [3, 'job id', 'text'], [3, 'job title', 'text'], [3, 'min salary', 'number'], [3, 'max salary', 'number'], [4, 'employee id', 'number'], [4, 'first name', 'text'], [4, 'last name', 'text'], [4, 'email', 'text'], [4, 'phone number', 'text'], [4, 'hire date', 'time'], [4, 'job id', 'text'], [4, 'salary', 'number'], [4, 'commission pct', 'number'], [4, 'manager id', 'number'], [4, 'department id', 'number'], [5, 'employee id', 'number'], [5, 'start date', 'time'], [5, 'end date', 'time'], [5, 'job id', 'text'], [5, 'department id', 'number'], [6, 'location id', 'number'], [6, 'street address', 'text'], [6, 'postal code', 'text'], [6, 'city', 'text'], [6, 'state province', 'text'], [6, 'country id', 'text']], 'foreign_keys': [[5, 1], [20, 10], [24, 6], [28, 10], [29, 6], [25, 14], [35, 3]], 'primary_keys': [1, 3, 6, 10, 14, 25, 30], 'table_names': ['regions', 'countries', 'departments', 'jobs', 'employees', 'job history', 'locations']}\\\n",
    "    \\n  #受众#\\n\\\n",
    "    Mysql数据库\\\n",
    "    \\n  #输出#\\n\\\n",
    "    只输出SQL查询语句\\\n",
    "    \\n  #目的#\\n\\\n",
    "    将问题\\\"What is all the information about employees who have never had a job in the past?\\\"转化为SQL查询语句'\"\n",
    "\n",
    "# 获得模型模板\n",
    "model_type = ModelType.qwen2_5_1_5b_instruct\n",
    "template_type = get_default_template_type(model_type)\n",
    "model_id_or_path = \"./model\"\n",
    "# 初始化模型\n",
    "model, tokenizer = get_model_tokenizer(model_type, torch.float32, model_id_or_path=model_id_or_path, model_kwargs={'device_map': 'cpu'})\n",
    "model.generation_config.max_new_tokens = 128\n",
    "model = Swift.from_pretrained(model, ckpt_dir, inference_mode=True)\n",
    "import gc;_=gc.collect();\n",
    "template = get_template(template_type, tokenizer)\n",
    "# 执行推理\n",
    "response, history = inference(model, template, query)\n",
    "# 格式化输出\n",
    "print('输入：---------------------------------------------------------------------------------------------')\n",
    "print(query)\n",
    "print('期望输出：-----------------------------------------------------------------------------------------')\n",
    "print('''  SELECT * FROM employees WHERE employee_id NOT IN (SELECT employee_id FROM job_history)''')\n",
    "print('输出：---------------------------------------------------------------------------------------------')\n",
    "print(  response)\n",
    "print('END：---------------------------------------------------------------------------------------------')\n",
    "# 缓存清理\n",
    "del model,template,tokenizer;import gc;_=gc.collect();"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd6167c0-71af-4d72-b658-ea2957923549",
   "metadata": {},
   "source": [
    "#### 2.5.2 评测微调后的模型\n",
    "\n",
    "（运行以下代码前请修改ckpt\\_dir为在验证集表现最佳的微调模型路径）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57fc4a46-d65f-4c15-9791-d94929d557ba",
   "metadata": {
    "ExecutionIndicator": {
     "show": true
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "!mkdir -p eval_outputs\n",
    "%env LOG_LEVEL=WARNING\n",
    "!swift eval \\\n",
    "--ckpt_dir 'qwen2-7b-instruct/vx-xxx/checkpoint-xxx<请修改为lora微调后checkpoint位置>' \\\n",
    "--eval_dataset no \\\n",
    "--custom_eval_config 'resources/2_4/data/config_eval.json' \\\n",
    "--max_length -1 \\\n",
    "--system '' \\\n",
    "--infer_backend 'pt' \\\n",
    "--name 'pre_train_evaluation' \\\n",
    "--eval_output_dir './eval_outputs'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58bd7a43-8338-4698-a4a0-ba8f0c544b2f",
   "metadata": {},
   "source": [
    "完整参数列表请参考：[eval 命令行参数](https://swift.readthedocs.io/zh-cn/latest/Instruction/%E5%91%BD%E4%BB%A4%E8%A1%8C%E5%8F%82%E6%95%B0.html#id16)\n",
    "\n",
    "而我们把微调后模型的分数与之前的参考分数进行对比：\n",
    "\n",
    "|   |  qwen-plus  |  qwen2\\_5-1\\_5b-instruct  |  qwen2\\_5-1\\_5b-instruct(微调后)  |\n",
    "| --- | --- | --- | --- |\n",
    "|  rouge-l-r  |  0.5995804511766886  |   0.4730549779987757  |  0.6540312152096097  |\n",
    "|  rouge-l-p  |  0.5671844066005733  |  0.3572425303637290  |  0.6203511702379731  |\n",
    "|  rouge-l-f  |  0.5755829728881274  |  0.3907812431837991  |  0.6202922260470989  |\n",
    "\n",
    "可以看到，`Rouge-L`分数都有了非常大的提升，几乎翻倍，性能甚至已经超过了`qwen-plus`。说明经过微调后的模型NL2SQL的能力得到了大幅度的加强。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f6017cf-da26-4277-8fe4-0f8b825d2cf4",
   "metadata": {},
   "source": [
    "### 2.6 参数矩阵融合\n",
    "\n",
    "模型微调训练完成后，有两种方式可以使用微调后的模型：\n",
    "\n",
    "1.  在调用时动态加载微调模型。  \n",
    "    微调后获得的低秩参数矩阵只占20MB的存储空间，这个大小非常便于做增量发布和传播，这也是工程上常用的方法。需要注意的是，用哪个基础模型微调，在加载时就需要指定使用哪个基础模型。  \n",
    "    我们在前一小节中，已经通过指定`ckpt_dir`尝试了这种方法。\n",
    "    \n",
    "2.  将基础模型与微调得到的低秩参数融合，获得一个完整的、更新了参数的模型，再调用融合了的模型。\n",
    "    \n",
    "\n",
    "这里我们介绍第二种方法：融合“微调参数矩阵”与”基础模型参数矩阵“，将微调后的模型参数存储成一个完整的参数矩阵。\n",
    "\n",
    "通过`swift export`方法，传入微调模型的路径（建议传入`best_model_checkpoint`），即可得到融合后的模型。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee8c6bc4-f667-480f-9f6a-633568e50cb3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%env LOG_LEVEL=INFO\n",
    "!swift export \\\n",
    "--ckpt_dir 'output/qwen2_5-1_5b-instruct/vx-xxx/checkpoint-xxx<请修改为lora微调后checkpoint位置>' \\\n",
    "--merge_lora true"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee4c54c6-e63f-40bd-8b2c-c1614b031c63",
   "metadata": {},
   "source": [
    "日志展示了融合后模型的路径。融合后的完整参数矩阵默认存储在`checkpoint`目录下。（PAI实验环境完整的模型参数在：`output/qwen2_5-1_5b-instruct/vX-XXX/checkpoint-XX-merged`）。\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa5a96d1-c017-4377-9654-e8401509c3c1",
   "metadata": {},
   "source": [
    "## ✅ 本节小结\n",
    "\n",
    "在本节课程中，我们学习了以下内容：\n",
    "\n",
    "*   了解模型小型化的应用场景、常见方式及其限制。\n",
    "    \n",
    "*   学习了模型高效微调的几种常用方法，并重点对LoRA高效微调的过程和优点进行了讨论。\n",
    "    \n",
    "*   最后通过一个LoRA微调实验，我们简单地进行了一次完整的微调。\n",
    "    \n",
    "*   学习了基本的模型评估标准，并在微调前后进行了模型评测和结果对比。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a7a163c-c116-445f-8406-7c57fc1ddeb0",
   "metadata": {},
   "source": [
    "由于本课程第2\\_9章节\"部署模型到生产环境\"的内容涉及到模型部署，也需要GPU示例，因此你可以继续留在这个GPU实例中学习2\\_9的课程内容。在2\\_9的教程中，你将了解如何将本地模型（是否经过微调都可以）部署到阿里云ECS或PAI上。如果你要部署微调模型，需要部署参数矩阵融合后生成的完整参数矩阵（后缀带merged），格式为：output/qwen2\\_5-1\\_5b-instruct/v2-202xxxxx-xxxxxx/checkpoint-xxx-merged。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3181253a-c044-429e-a54b-adf917e10866",
   "metadata": {},
   "source": [
    "## 扩展阅读\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3355161-2313-495c-a28a-69f4af57ba5c",
   "metadata": {},
   "source": [
    "### 开展微调任务前必须要关注的\n",
    "\n",
    "虽然在本教程中你可以使用准备好的NL2SQL数据集，免费体验GPU算力资源进行微调，但在实际生产中，微调往往并不简单。高昂的算力成本与对数据规模和质量的严苛要求都是你在微调前需要考虑的因素。\n",
    "\n",
    "在开始微调任务之前，建议你关注以下方面：\n",
    "\n",
    "1.  明确业务需求匹配度  \n",
    "    首先，你需要清楚业务的具体需求，明确微调要解决的具体问题，并评估当前模型是否已能满足大部分需求，如果是，则可能不需要微调。确定是否有明确的业务指标来衡量微调前后的效果，例如NL2SQL的查询成功率。\n",
    "    \n",
    "2.  尝试更大的模型  \n",
    "    你可以考虑使用参数更大的模型，因为它们经过更多数据和训练轮次，在通用领域和特定领域都可能带来显著的性能提升。 即使有使用成本和延迟限制，也建议进行实验或性能验证。更大的模型在微调时可以作为“老师”，传授知识给较小的“学生”模型。\n",
    "    \n",
    "3.  使用提示词工程和插件调用  \n",
    "    这种方法比对模型进行微调更灵活、成本更低，并且这些前期工作可以在构建微调数据集时复用。\n",
    "    \n",
    "4.  评估资源和技术可行性  \n",
    "    微调过程需要计算资源和时间成本，包括GPU资源（训练时需要）、存储空间（存储模型时需要）和专家人力成本（构建数据集时需要）。评估项目预算和资源是否允许进行有效微调，同时确保团队具备所需的技术能力，或寻求合作伙伴提供支持（你也可以直接使用[百炼提供的0代码微调能力](https://help.aliyun.com/zh/model-studio/user-guide/using-fine-tuning-on-console)）。\n",
    "    \n",
    "5.  确保充足的数据  \n",
    "    虽然微调相比于预训练阶段对数据要求降低，但仍需准备足够的精确标注数据集，通常需要至少1000条数据才能获得良好效果。\n",
    "    \n",
    "6.  关注合规与隐私  \n",
    "    确保使用的数据符合法律法规要求，处理个人数据时遵循隐私保护原则。\n",
    "    \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b242ed7-4c2d-4bdb-9e9a-7d5403747f02",
   "metadata": {},
   "source": [
    "### 微调数据集构建策略\n",
    "\n",
    "一般来说，在比较复杂的场景中，微调至少需要**1000+条优质的训练集数据**。构建数据集时，请确认以下几点：\n",
    "\n",
    "*   **数据质量**：确保数据集的准确性和相关性，避免模糊和错误内容。\n",
    "    \n",
    "*   **数据多样性**：覆盖任务的所有关键方面和潜在变化，包括不同场景、语境和专业术语。\n",
    "    \n",
    "*   **平衡性**：如果任务涉及多种类别场景，确保各类别样本均衡，防止模型偏向于某一类。\n",
    "    \n",
    "*   **持续迭代**：微调是一个迭代过程，根据模型在验证集上的表现反馈，不断优化和扩大数据集。\n",
    "    \n",
    "\n",
    "而如果你在进行模型微调时缺乏数据，建议你使用知识库检索来增强模型能力。\n",
    "\n",
    "> 在很多复杂的业务场景中，可以综合采用模型调优和知识库检索相结合的技术方案。\n",
    "\n",
    "你也可以采用以下策略扩充数据集：\n",
    "\n",
    "1.  联系数据专家或数据团队基于已有数据，制作更多的典型场景数据。\n",
    "    \n",
    "2.  让能力更强的大模型模拟生成特定业务/场景的相关内容，辅助你生成更多可用于微调的数据。\n",
    "    \n",
    "3.  通过应用场景收集、网络爬虫、社交媒体和在线论坛、公开数据集、合作伙伴与行业资源、用户贡献等方式获取更多数据。\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b1ab94c",
   "metadata": {},
   "source": [
    "## ✉️ 评价反馈\n",
    "感谢你学习阿里云大模型ACP认证课程，如果你觉得课程有哪里写得好、哪里写得不好，期待你[通过问卷提交评价和反馈](https://survey.aliyun.com/apps/zhiliao/Mo5O9vuie)。\n",
    "\n",
    "你的批评和鼓励都是我们前进的动力。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e76c79e-cb06-4a10-b6f0-57cc84ed069d",
   "metadata": {},
   "source": [
    "## 🔥 课后小测验\n",
    "\n",
    "【单选题】2.4.1. 以下关于 LoRA 的描述，哪一项是错误的？（ ）\n",
    "\n",
    "A. LoRA 可以有效降低微调大型语言模型的成本。\n",
    "\n",
    "B. LoRA 会修改被微调模型的原始权重。\n",
    "\n",
    "C. LoRA 的实现相对简单，易于集成。\n",
    "\n",
    "D. LoRA 微调的结果可以方便地回退。\n",
    "\n",
    "答案：B\n",
    "\n",
    "解析： LoRA 并不直接修改原始权重，而是通过添加低秩矩阵间接影响模型行为。 这使得回退操作变得简单，只需移除添加的低秩矩阵即可。\n",
    "\n",
    "【多选题】2.4.2. 你正在使用 Swift 微调一个 Qwen 模型，发现模型在验证集上的 loss 出现了明显的上升趋势，以下哪些操作可以帮助你缓解或解决这个问题？( )\n",
    "\n",
    "A. 增大 --learning\\_rate\n",
    "\n",
    "B. 减小 --learning\\_rate\n",
    "\n",
    "C. 增大 --lora\\_dropout\n",
    "\n",
    "D. 减小 --lora\\_dropout\n",
    "\n",
    "E. 增大 --num\\_train\\_epochs\n",
    "\n",
    "F. 减小 --num\\_train\\_epochs\n",
    "\n",
    "答案: B, C, F\n",
    "\n",
    "解释:\n",
    "\n",
    "*   learning\\_rate: 学习率过大会导致模型训练速度快，但可能在最优解附近震荡，甚至无法收敛，导致loss波动，看起来像是过拟合。当然这与真正的过拟合不同。\n",
    "    \n",
    "*   lora\\_dropout: 增大 dropout 率可以增强模型的泛化能力，抑制过拟合。\n",
    "    \n",
    "*   num\\_train\\_epochs: 过拟合也可能是训练次数过多导致的，减少训练次数可以避免模型过度学习训练数据。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
